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Article history: Road users make vital decisions to safely maneuver their vehicles based on 
the road markers, which need to be correctly classified. The road markers 
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corresponding two features are determined. It is observed that the achievable 
accuracy of the proposed methods at several non-urban road scenarios is 
approximately 96% and the processing time per frame is significantly 
reduced when the video resolution increases as compared to that of the 
existing approach. 
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1. INTRODUCTION 

The number of road accidents in Malaysia increases at an alarming rate of 9.7% every year over the 
past thirty years [1] contributed by (road users) errors, faulty road environment, and vehicle defects. The road 
marker, is very important in guiding road users while driving. Different types of road markers in non-urban 
roads such as double solid (DD), dashed (D), solid-dashed (SD), dashed-solid and single solid (SS), allow 
drivers to safely decide either to maintain the course in the middle of the lane or to overtake the front 
vehicles. In earlier works, many efforts were invested in solving lane detection and tracking problems for 
Auto-Assist Driving System (ADS) [2], [3]. These methods had been used as a warning system to the driver 
[4]-[8] and surprisingly, the lane detection also was used as part of the system to analyse the driver behaviour 
[9]. Symbols, alphabets, and custom markers were also being explored and used as the information to alert 
drivers [10,11]. 

However, road marker classification [12], [13] still remains an open question due to the varying 
types of road markers across the globe with limited effort being put to solve the problem in road marker 
classification. Collado et al. [14] proposed a frequency analysis method for the urban road marker 
classification, which uses the inverse perspective mapping (IPM) to generate a bird-eye view of the road and 
produce a modified Hough Transform (HT) for better lane detection. The method then applies power 


Journal homepage: http://iaescore.com/journals/index.phpNJECE 


Int J Elec & Comp Eng ISSN: 2088-8708 O 2541 


spectrum and Fourier analysis to detect three types of markers, which are; single solid, dashed, and merged 
road markers. However, road marker classification for non-urban type roads was not addressed. 
Schubert et al. [15] studied lane changing based on road marker recognition surrounding traffic situation but 
the road markers classified were limited to only; dashed and solid lines. In a work by Lindner et al. [16], an 
edge detector technique was designed to subsequently search for a group of four objects namely the lines, 
curves, parallel curves and closed objects in detecting road markers. Although four types of road markers 
were classified, most of the road markers are dashed lines with different sizes. 

Meanwhile, another research from Suchitra et al. classified three road markers namely; dashed, solid 
and zigzag using a modular approach [17]. The road markers are classified as either dashed or solid using the 
Basic Lane Marking (BLM), which is based on continuity properties. This approach however, applies the 
temporal information in the classification operation, which renders slower detection when the road marker 
type changes whilst driving on the road. In addition to these methods, Nedevschi et al. [18] uses a periodic 
histogram to determine the type of road markers. This ego-localization is observed to enable the classification 
of road markers into four types namely single solid, double solid, dashed, and merged. 

In a recent research, Paula et al. [19] presented an automatic classification technique to classify five 
types of road markers. The approach uses between three to five features extracted from the image, which is 
later fed to the artificial neural network. A two-stage method is modelled for the full classification, with the 
first stage applies the Bayesian classifier for dashed, single dashed and double solid while the second stage 
differentiates between dashed-solid and solid-dashed lines for road marker detection. However, the results of 
the classification were found to have changed abruptly on each frame, which caused inconsistent results 
while driving. Mathibela et al. [20] proposed a new approach using a unique set of geometric features which 
functions within a probabilistic RUSBoost and Conditional Random Field (CRF) network to classify the road 
markers into seven types, including; single boundary, double boundary, separator, zig-zag, intersection, 
boxed junction and special lanes. Even though more types of road markers were classified using this method, 
unfortunately, the classification only used static images in urban roads. 

On the basis of the comprehensive literature review, the extensive classification of road markers had 
been rarely studied. It is interesting to look at the aforementioned different approaches for road marker 
classification although no standard databases available to allow fair comparison. In addition, the dimension, 
colour and size of the markers are varied across the world. 

This paper proposes a novel approach to classify these road using a customized Region of Interest 
(ROTI) in a video acquired from a camera with its calibrated position. Two features are derived, namely; the 
contour number, Nc(t), and the contour angle, @, are later fed into a two-layer classifier. The first layer 
classifies the D and SS marker types based calculated Nc(t), values, while the second layer classifies the 
DD, DS or SD marker types, by using @ values as shown in Figure 1. Temporal information integration is 
applied to improve the classification accuracy by validating the marker type’s transitions on the road. This 
algorithm has been demonstrated an overall accuracy of approximately 95% and reduced reduction in the 
processing time by ~50% shorter than the existing method. 


SERE 


Dashed (D) Dashed-Solid (DS) Double Solid (DD) Solid-Dashed (SD) Single Solid (SS) 


Figure 1. Road markers found at two-way non-urban road 


2. RESEARCH METHOD 
2.1. Camera setup for region of interest selection 

Many existing methods for selecting the ROI such as the vanishing point, area-based detection and 
area-based tracking methods had been used in the past. A technique of calculating the vanishing point using 
Hough Transform (HT) is carried out by obtaining the intersection line and the ROI at the lower half of the 
image [21]. In [22], [23], the ROI is selected over the whole horizontal axis and a limited range along the 
vertical axis, before the feature extraction and road marker or lane classification take place. 

In this project, a new method in selecting the ROI based on its calibrated camera’s height and Field 
of View (FOV) is proposed. For the initial setup, a camera with a resolution of 1280x720 located at the 
center of the car is calibrated on its position with its FOV adjusted towards the planar road surface captured 
by the camera as shown in Figure 2(a) and Figure 2(b). To calibrate the position of the camera, two 
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horizontal lines, A7 and h2, are set on the FOV to equally divide into three sections as in Figure 2(c), where 
the h1 horizontal line in the image must be at the same level with the most right lane in the image. 


(a) (b) (c) 


Figure 2. The camera setup (a) Camera position (b) 3d camera coordinate system (c) Camera height 
adjustment with h1 and h2 horizontal lines 


The image captured is then divided into 40 equally sized sections or squares, each of which having a 
size of 160x144 pixels as shown in Figure 3(a) and Figure 3(b). The ROI selection method is applied to every 
video frame to choose rj (x,y) as the ROI, which contains the road marker, as shown in Figure 3(c). It can be 
observed that r{(x,y) is the nearest to the car where the effects of the road curve on the road marker is low. In 
the case of lane departure, the ROI will contain no marker for a longer duration which indicates that the 
vehicle is departed from the right track. In our approach, r;, containing the RGB information, is converted to 
greyscale and filtered through Gaussian filter [24] before undergoing the thresholding process using Otsu 
Thresholding method [25] for binary conversion. Then the two features, which are the contour number and 
the angles, will be extracted to complete the classification as discussed next. 
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Figure 3. (a) Row and column post camera position calibration (b) ROI r1 (c) ROIs consist of road markers 


2.2. The number of contours, Nç(t), and the angle @ in Method A and B 

A contour is a line that connects all points along the boundary of an object of the same colour. The 
contour number, Nc(t) in the ROI is calculated by counting the number of all contours along its horizontal 
axis. The calculation of Nc(t) is carried out in the ROIs, r{(x,y), for every frames in a complete cycle, which 
is defined as the time taken by all consecutive frames containing a complete road marker pattern. This pattern 
will repeat itself after every complete cycle until the road marker type changes or ends as in Figure 4. 

The proposed road marker classification is based on the contour number, N-(t), and the contour 
angle, 0, implemented using two methods of two-layer classification as depicted in Figure 5. The formulation 
of Nc(t) and 0 for both of these methods, which are named as Method A and Method B, will be presented 
next. 

In Method A, road marker D is detected when the minimum Nc(t) over the cycle is zero and the 
maximum N(t) is one. If Nc(t) = 1 over a complete cycle, then the road marker is SS. Otherwise, if the 
maximum Nc(t) over the cycle is two, then the second layer classification using the angle 8 of the centroids 
will be run to classify the marker as SD, DS or DD. The angle 8, which is measured between a line 
connecting the two centroids and the horizontal axis, is calculated only on the first frame in a set of frames 
showing a particular road marker pattern that repeats itself after the last frame in the set before the pattern 
changes. This can be identified from the change in Nc(t) value, which is from 140 or 12. There are three 
threshold angles 6iy1,9n2 and O@n3, which are determined from training dataset, applied for the 
classification. If O43 < @ < @p2, then the road marker is DS. If 642 < 0 < Oiy1, then the road marker is 
SD and if O < Oth or 0 > Oina, then the road marker is DD. 
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(a) (b) 


Figure 5. Two-layer classification diagram (a) Method A (b) Method B 


In Method B, the significant different is when detecting DD. Instead of detecting DD at the second 
layer using the angle 6, DD is detected at the first layer based on Nc(t), when it is equal to two over a 
complete cycle. Hence, only one threshold angle, @,n,2, used to detect DS and SD. In order to find the contour 
angle, as seen in Figure 6 (d) and formulated as follows: 
oe! Y27-V1 
0 = arctan e] (1) 


27X1 


(r) 
Two centroids, (x1, y1) and (x>,y2), are calculated from the moments of the markers, x = ST 
00 


surrounded by the contours as shown in Figure 6(a) and Figure 6(b). The red line represents 


(r) 
(r) = mor 
= moo)’ 


the contours and the red dots denote the centroids. The moments are calculated as 
mj = eee contour r / (x) ¥)x/y', where r represents the region in the contour (the red lines) [26]. In the 


next section, the threshold angles 6,,; will be determined to classify the road marker as DD, DS or SD (for 
Method A), where i=1,2,3 for Method A and only 6,2 is used for Method B. 
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(a) (b) (c) (d) 


Figure 6. (a)-(c) ROI with centroids found (d) @ value calculation from two centroids 


2.3. The threshold angles 6,, ; formulation for the second-layer classification in method A and B 

In Method A, the threshold angles 0,,,; are formulated based on the angles 6 between two centroids 
of each of the 300 frames extracted from four video clips (Vid22_USB, Vid24_USB, Vid26_USB, 
Vid27_USB) as in Table 2. The angles @ are then plotted accordingly in Figure 7. As for DD, the @ values 
are found to be distributed between the negative angles (denoted as DDlower) and the positive angles 
(denoted as DDupper) due to the change in the location of the two centroids shifting along a curvy road. The 
angles 0 for the 1“ frames in every set of frames for DS, SD and DD are found to be separated with each 
other, as seen in Figure 7. It is clear from this figure that the threshold angles, 6,,;, are the separating angles 
or lines between four groups of angles, where i = 1,2,3 for Method A. 


6 /angle Individual Value Plot of DS, SD, DDupper, DDlower 
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Figure 3. Distribution of the @ for On; 


The three threshold angles, Oth, Oth,2 and @:,3, which are represented by the blue horizontal lines 
in Figure 7, are calculated such that the angles or 0 values are separated or classified intro three types; 
namely DD, SD and DS. Hence, these threshold values are the midpoint between the maximum angle of a 
marker type and the minimum angle of the next neighbouring marker type. The minimum and maximum 
angles of DDupper, SD, DS and DDlower are stored as the first row (minimum angles) and the the second 
row (maximum angles) of matrix © respectively. Based on the @ values recorded in Figure 7, the 
corresponding matrix © is given below. 


DDu SD DS DDI 
_ ( 63.8° 24.3° —8.0° 89.5° \Min 
e= ( ) 


89.9° 41.2° 10.5° —25.1° / Max 
where the threshold angles are calculated using the following formula 


— 92,i41 + 91,5 


ie (2) 
with O41 = 52.5°, Oth,2 = 17.4° and Oth = —16.5°, as shown in Table 1 (a) for making the classification 
decision. As for Method B, only 0,2. = 17.4° is applied for the classification decision, as shown in 


Table 1 (b) below. 
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Table 1. 8 value for (a) Method A and (b) Method B 


Marker Threshold ( 8 ) Marker Threshold ( @) 
DD 8 > 52.5° OR 6<-16.5° @>17.4° 
sD 17.4° < @<52.5° DS 6<17.4° 
DS -16.5°<6<17.4° 

(a) (b) 


2.4. Resolving errors in classification 

Errors in road marker classification tend to happen during the marker transition period. The frame 
delay time might also cause errors in classification. Both of these cases can be solved by using the temporal 
information between the classification results during the transition period. The markers tend to be similarly 
labelled for a set of adjacent frames. Knowing this tendency, the previous set of classification result will be 
stored and if there is a transition of marker types, it will wait on the next set of frames classification results 
for a final validation to avoid unnecessary classification errors. 


3. RESULTS AND ANALYSIS 

The proposed algorithm was tested with multiple video sequences captured by two different devices 
with different car velocities using a lower range USB camera (Logitech c310) and a higher range camera 
(GoPro Hero Silver). To evaluate the robustness of the new algorithm, other sets of videos acquired with 
different devices at different resolutions and the velocity of the vehicle is between 60-90 km/h for the test 
video images, which meets the speed limit in the suburban road area. The videos are also taken when the 
vehicle was moving on different road conditions including flat, hilly and narrow roads. The videos used to 
determine the threshold angles were taken with a moderate vehicle speed between 50-60 km. The 
experiments were carried out using a desktop computer equipped with Intel® core ™ i7-4790 CPU 
@360GHz and 16 GB RAM. The proposed system was implemented by using the Visual Studio C++ 2010 
compiler with the library from OpenCV version 2.4.11. 

In the classification process, if Nc(t) = 1, the software will classify it as either D or SS depending 
on Nc(t) found in the next set of frames. For example, if the next set of frames detected with Nc(t) = 0, the 
system will classify it as a D types whereas if N¢(t) = 1 consistently detected over the complete cycle, the 
system will classify it as SS types. If the maximum N_(t) value is two over the complete cycle, the road 
marker will be classified as either DS, SD or DD, by using the 6 value calculated at the 1“ frame from the set 
of frames. The classification result will remain and will only be reclassified if N¢(t), changes from 2>1. For 
the case of DD type which expects Nc(t) = 2 consistently, a sampling up to its 10th frames will be 
performed when only two contours are detected and classification result will be updated by using the @ value 
on its 11" frame. 

The accuracy of the marker classification is calculated using Equation (3). The total sets of frames 
classified correctly, Xp, will be divided by the total sets of frames in the video image XE, excluding the total 
number of 1“ sets of frames when the road marker transition happens, £t1. The exclusion of =t1 is due to the 
road marker transition in which the angle @ calculation is temporarily paused. The clips and the accuracy 
results are shown in Table 2. 


Accuracy = ee X 100% (3) 


As shown in Table 1, clips 4 to 7 were trained for identifying 04,,. Clips 9 and 10 are recorded 
when the vehicle velocity is between 60 and 80 km/h. The remaining test clips contain at least two types of 
road markers with different resolutions and frame rates. It can be observed that the accuracy achieved by 
Method B is better compared to method A as only one threshold angle is used. The processing time per frame 
for Method B is slightly lower on average as compared with Method A. It can also be seen in Table 1 that 
different types of video compression formats like .wmv renders a longer processing time, as shown for clip 6 
and 7. Our method is also tested against the existing approach presented by Paula at [19] using a few videos 
with different resolutions. The resulted average processing time per frame is recorded in Table 3. Our 
approach has been observed to perform better at larger resolutions. This is contributed mainly by the smaller 
ROI applied in our approach, besides the two formulated features namely the contour number and angle used 
to classify the road marker types. 
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Table 2. Clips Information and the Accuracy Results for both methods 


Tete mt of Misia namn Method A | Method B 
Frames Resolution Lane Marker Tossa for] Test Time per | Time per 
Gch a Frame (ms) | Frame (ms) 


Test_Res_640-1.m4v 640 x 480 
Test_Res_640-2.m4v 640 x 480 
Test_Res_640-3.m4v 2 640 x 480 
Vid22_USB_60.mp4 2 3 1280 x 72 

1280 x 72 


1280 x 72 

1280 x 72 

1280 x 72 
Test_GoPro_60KM1.mp4 7 1280 x 72 
Test_GoPro_S0KM2.mp4 757 25 1280 x 720 
Test_GoPro_Road3.mp4 2 7 1280 x 720 
Test_GoPro_Road6.mp4 22 1920X1080 
Test_GoPro_Road7.mp4 2 1920X1080 

*DD=double solid, SD=solid-dashed, DS=dasshed-solid. D=dashed 


# Accuracy is total of correct set of markers classified correctly from all set marker in the video, excluding the first set of marker when markers types transitioning 
^ Video with DD, SD or DS consist of either right or left curve road situation 


MMMM OM OM OM OM OM OM OO E 


Table 3: Average Time Execution for Each Frame 


Resolution Size Average Time per frame (ms) 
M. B. de Paula and C. R. Jung (15) New proposed Method 
640X480 3.71 7.1 
1280X720 6.53 8.78 
1920X1080 16.59 11.19 


4. CONCLUSION 

A novel approach on the road markers classification process is presented in this paper. A smaller 
ROI is used to process the video frames and extract the formulated features, which are the contour number 
and angle, as the inputs for the proposed two-layer classifier algorithm. The average accuracy of ~96% has 
been achieved using our proposed algorithm, with a relatively lower processing time per frame at large video 
resolution as compared with the existing approach. The errors found in the road marker classification are 
caused by the abrupt changes in illumination, vanishing road marker, white elements on the road and markers 
blocking by the other vehicles. Future work will focus on the illumination and faint marker issues that affect 
the features extraction for classifying the road marker types. 
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